Calibration with Many Checking Rules

Each period an outcome (out of finitely many possibilities) is observed. For simplicity assume two possible outcomes, a and b. Each period, a forecaster announces the probability of a occurring next period based on the past. Consider an arbitrary subsequence of periods (e.g., odd periods, even periods, all periods in which b is observed, etc). Given an integer n, divide any such subsequence into associated sub-subsequences in which the forecast for a is between [ i n , i+1 n ), i ∈ {0, 1, .., n}. We compare the forecasts and the outcomes (realized next period) separately in each of these sub-subsequences. Given any countable partition of [0, 1] and any countable collection of subsequences, we construct a forecasting scheme such that for all infinite strings of data, the long run average forecast for a matches the long run frequency of realized a0s. ∗Sandroni and Smorodinsky thank the financial support from the NSF Grant SBR 9730385, BSF Grant 97-00113/1 and the Bergmann Memorial Research Grant †Department of Economics, University of Rochester, Rochester NY 14627 ([email protected]) and Department of Managerial Economics and Decision Sciences, Kellogg School of Management, Northwestern University, Evanston IL 60208. ([email protected]) ‡Davidson Faculty of Industrial Engineering and Management, Technion Haifa 32000 Israel. ([email protected]), and Department of Managerial Economics and Decision Sciences, Kellogg School of Management, Northwestern University, Evanston IL 60208. §Department of Managerial Economics and Decision Sciences, Kellogg School of Management, Northwestern University, Evanston IL 60208. ([email protected])